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Response to Reviewers Comments: 


Reviewer: A 

General Comment: [| read this paper as a neuroscientist who might be part of the target audience. My 
impression is that it presents a very simplified (compact) overview for how to conduct a systematic review 
and meta-analysis (SRMAN). I very briefly reviewed the two papers that it is based on (Vesterinen et al. 
2014 and Muka et al. 2020) and it does seem to follow and synthesize the process outlined in those papers, 
but in an extremely simplified form. It could therefore be useful a starting point for a researcher interested 
in conducting SRMAN but I suspect its main utility is in pointing the reader towards more comprehensive 
resources. 

#General Response: Thanks for all your comments. We tried to incorporate all your suggestions in our new 
version of the manuscript. 


Major concerns and questions 

1. I wonder if this paper really does enough that is distinct from Vesterinen et al. (2014) to warrant 
neuroscientists starting here, rather than referring directly to Vesterinen (for animals) or maybe Muka (for 
humans). To me, the advice contained in this guide all seemed quite general and applicable to virtually any 
field. I think there needs to be some additional clarity as well about whether this for neuroscience generally 
or for animal studies only (implied by references to "basic neuroscience" but I don't think this automatically 
means animal-only). 

Response: Thanks for your comments. We have restructured the entire manuscript intending to clarify these 
issues. We added references and examples applicable to neurosciences in the protocol elaboration phase. 
For example, see the item “Screening or selection strategy”; “See examples of protocols presenting 
screening strategies in the field of neuroscience by Hohls (37), Ramos-Hryb (33) and Bolzan and Lino de 
Oliveira (34)”. 


2. Thinking about my own field within neuroscience, much of the work is exploratory and not closely 
replicated. A particular mechanism is rarely manipulated the same way for the same outcome measure. 
How much can or should we collapse things together conceptually? For example, is it OK to meta-analyze 
across similar but distinct outcomes (e.g. 4 different measures of stress)? What about if the mechanism is 
close (e.g. the same drug or drug-class, but injected into different brain regions) while the outcome measure 
is the same? 

Response: Thank you for your comment. It is typical for studies in this area to show results similar to what 
you could do. In this case, one possibility to investigate an effect size in heterogeneous ones is to stratify 
in subgroup analyses. We mentioned this in the item "Meta-analysis". 


3. A few times within the paper, references are made to collaborations with librarians or statisticians. I felt 
like this was a little vague. For example, can a librarian really help me with elaboration of a research 
question? Or should I really be seeking their advice more specifically for assistance with scholarly 
databases and reference management software? 

Response: Thank you for your comment. In our experience librarians provided good instruction how to 
prepare a searchable question using mnemonic tools (like PICO) and use bibliographic databases and 
reference management (we added a remark about it in the topic "Search strategy" in the section: Phase 2). 


4. Many acronyms are described (PICO, ECLIPSE, SPICE, SPIDER), but it seems only in passing. What 
are the relative advantages and disadvantages of these approaches? For example, is, say, SPIDER, better 
for someone trying to do an SRMAN on human EEG data? Or how suitable would ECLIPSE be for 
neurobiological results from animal studies? 

Response: Thank you for your comment. The mentioned the use of ECLIPSE, SPICE, SPIDER approaches, 
besides PICO and variations, to provide flexible alternatives to the reader. We agree that the information 
was left vague when its advantages and disadvantages. These can be useful when specific types of research 
(e.g., in vitro, in vivo, ex vivo, or brain imaging) are the main interest of the review. Therefore, we have 
added this information. 


5. I found the structure of the article a little confusing. The main description of each phase was quite brief, 
but the "Brief description of steps encompassing the five phases of SRMAN" was, in contrast, quite long. 
Maybe a restructuring of the paper is in order so that more detail is provided in the main text and brief 
descriptions or summaries are presented in another way (e.g. text box or summary table). 

Response: Thanks for your comment and suggestions. We have restructured the text completely by adding 
more details in the phase 2 (elaboration of a review protocol). Also, we've added 3 tables: 

Table 1, for the PICO tool: categories, definitions and examples. 

Table 2, for the description of a generic search strategy elaborated to obtain relevant publications to answer 
a hypothetical review question created using the PICO tool. 

Table 3, for the description of generic inclusion and exclusion criteria to a screening strategy of studies 
relevant to answer a hypothetical review question created using PICO tool. 


Minor concerns and questions 

1. One thing I appreciated in Vesterinen is how they noted which software packages were proprietary. I 
recommend that the authors highlight, in particular, which software packages are free (and open source) to 
help the budget-constrained neuroscientist more quickly identify the best approach. 

Response: Thanks for your suggestion. We make suggestions of selected free resources to implement each 
step of an SRMAN in any field of nonclinical research, including neurosciences. We have added this 
information in the section: Phase 4 (protocol implementation). 


2. In Phase 3 it is suggested that PROSPERO is the "most important public platform accepting registrations 
for rapid reviews, umbrella reviews, and systematic reviews..." Why is this? Is it because of the 
preregistration templates or because this is the most frequently cited location for preregistered SRMANs 
(and related review types)? Can other platforms, like Open Science Framework (osf.io) be used just as 
well? 

Response: Thanks for your comment. We cite PROSPERO as the most important platform for review 
registration because it is a large international database of systematic reviews used for registries in different 
areas: health, social care, welfare, public health, education, crime, justice, international development and 
others. In other words, PROSPERO concentrates most of the records of systematic reviews, but the Open 
Science Framework (OSF) can also be used. We have added this information in the last paragraph of 
section: phase 3 (protocol preregistration). 


3. I think the paper might benefit from sketching out an example situation. This would help me to better 
understand exactly what each of the phases involves and how it might look in my own neuroscience 
research. 

Response: We agree with your comment and have added some examples to the neuroscience-focused 
handwritten text. For example, definitions for population, intervention, control, and outcomes commonly 
used in this area, practical examples of protocols, and others. In addition to rearranging the text, we have 
provided a figure (Figure 1) with an illustration of all the steps. We hope you agree with what you thought. 


Reviewer: B 

I am sympathetic to the need for clear guidelines and how-to manuals for systematic reviews and meta- 
analyses. I don't see anything incorrect in the present manuscript; however, I don't understand the goal of 
the manuscript. The authors present multiple tools that can be used for SRs and MAs and described what 
an SR/MA consists of. There is very little in the manuscript specific to neuroscience. The title says the 
manuscript is a "guide" and the abstract states "These methods involve individually simple decisions, which 
may become complex when considering the whole process.". In its current form, I feel the manuscript is 
poorly written as a guide and that it makes SR/MAs seem very complex because the authors describe all 
the information in the main text and don't organize information into tables, steps, diagrams, or resources 
(beyond the 5 overarching steps they describe). I feel the manuscript could be published with minor edits, 
but has limited value. Reorganizing the information into amore "guide" format that has step-by-step 
instructions and is specific to basic neuroscience might be more useful. However, there may already be 
more established and thorough guidelines that exist (e.g., Cochrane, Johanna-Briggs Institute, PRISMA, or 
others). I would recommend systematically searching what guidelines already exist and then organizing 
this manuscript so that it adds value beyond what already exists (or if this has already been done, explaining 
more clearly what value this manuscript adds). Although, I leave it to the editors and authors to decide how 
to move forward. 

Response: Thanks for your comments. We agree with your comments and have restructured the entire text 
of this manuscript. We hope it's better and clearer now. We better detail the phase involved in the process 


of elaborating the research question, elaborating the protocol, preregistration, implementing or executing 
and disseminating the data. We added tables for the PICO tool: categories, definitions and examples; for 
the description of a generic search strategy elaborated to obtain relevant publications to answer a 
hypothetical review question created using the PICO tool; and, for the description of generic inclusion and 
exclusion criteria to a screening strategy of studies relevant to answer a hypothetical review question created 
using PICO tool. We agree and find your suggestions helpful on reorganizing the information and 
highlighting the purpose of this manuscript for the field of neuroscience. For this, we add examples of 
protocols published in this area that can serve as examples for some steps in the preparation and execution 
of a systematic review and meta-analysis. 


